Architectures based on artificial neural networks (ANN) are increasingly being used as preferred alternatives to von Neumann or Harvard style processor architectures for many computational applications. Indeed, ANN architectures are already used in many applications such as pattern recognition, voice and image recognition, business analytics, safety and security, and so on. ANNs typically comprise interconnected sets of nodes that can act like and are often referred to as their biological analogs, “neurons.” Each neuron has an output, frequently referred to as an “axon,” and one or more inputs provided via connections often called “synapses.” The synapses typically have associated weights or coefficients which amplify or otherwise modify the signals carried by the synapses and/or received by the neuron. The neuron typically sums the signals of all “upstream” connected synapses (i.e., synapses that provide inputs to the neuron) and any bias applied to the neuron. Through an activation function, the neuron changes the output axon, which may then be provided via one or more “downstream” synapses to subsequent neurons or as an output of the ANN. The result is a network that can process incoming signals and drive output signals.
Mathematical representations of ANNs have been implemented in software with some success and are used in areas such as object detection, voice recognition, and data mining applications, among many others. Software-implemented ANNs are dynamic in that they can be “trained” to solve many different problems. The ANNs can be trained using techniques such as supervised training, where datasets are provided that include objects, voices, images, and so on. Software implemented ANNs can be more efficient than traditional coding algorithms, particularly in terms of the ability of the ANN to be modified or retrained; but there is still a significant gap in performance of a trained ANN relative to an optimal ANN. A common way of implementing a high-performance ANN in a semiconductor technology is to train a specific network for a specific task, and then hard-code that solution directly into the semiconductor technology. While this technique can produce high computing efficiency for a particular implementation, it further results in a subsequent inability to reconfigure the network by changing weights, biases, or interconnections between neurons, or by adding or removing neurons. Other ANNs use memory elements such as SRAM or volatile registers to create an ANN that can be programmed. However, these memory elements are physically large, they limit the feasible size of the ANN, and they may also limit the flexibility of the connections between neurons.